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A statistical analysis was conducted to investigate and 
evaluate the relationship of English comprehension levels 
of foreign students with their graduate academic performances 
at the Naval Postgraduate School. 

In this study, English comprehension level test scores 
of foreign students were correlated with the final academic 
grades received at the school. 

The results obtained revealed a positive, but marginally 
useful, relationship between the two variables. The results 
were also analyzed by curricula subgroups and by military 
rank. 
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I. INTRODUCTION 



A. GENERAL 

Many factors determine the academic achievement of a 
student. If the learning situation is in a foreign country, 
adequate communication capability would be a most critical 
and basic requirement for successful performance by foreign 
students. 

This study was intended to investigate and evaluate how 
the English language comprehension levels of foreign stu- 
dents from countries where English is not the principal 
language are related to their academic performance at the 
Naval Postgraduate School (NPS) . 

This study will also provide an occasion to examine the 
effectiveness of a current English comprehension level test 
in predicting academic grades in the postgraduate study at 
the NPS. 

B. DEFINITIONS 

1. Foreign Student - A person from a country where 
English is not the principal language and who is required 
to take a English comprehension level test to study at NPS. 

2. Quality Point Rating (QPR) - A student's overall 
academic grade average computed from the numerical standards 
established by the NPS. (See Chapter III.B) 

3. English Comprehension Level (ECL) - A numerical 
grade of English proficiency converted to qualification 
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levels, as established in a numerical system by the Defense 
Language Institute (DLI) to show the English language 
capability of non-English speaking nationals. (See Chapter 
III.B.) 

C. VALUE OF THE STUDY 

All foreign students are required to have a certain 
English language capability in order to be admitted to the 
schools in the United States, but the facility with which 
one communicates in a foreign language rarely equals that 
in his native tongue. Hence, one can expect English 
language deficiencies to be a problem among foreign students. 

Without a sincere understanding of the language problem, 
there might exist not only an unintentional collision of 
feelings, but also discrimination in evaluating students. 
Grades may sometimes be inflated due to the tendency of 
professors to be quite lenient in evaluating foreign stu- 
dents, or sometimes grades may be deflated because the 
same norms may be used for all students. 

The findings of this study may suggest some ideas for 
school authorities in setting up policies concerning for- 
eign students, and for the agencies responsible for the 
English proficiency test to review the measures of English 
capacity of foreign students admitted to various levels of 
schools in the United States. 

Further, this study may help individual foreign coun- 
tries examine their screening procedures for their candi- 
dates to enter NFS. 
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This study will also provide some basis for further 
study in determining factors predictive of academic achieve- 
ment of non-English speaking nationals at the NFS. 
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II. STATEMENT OF THE PROBLEM 



A. THE PROBLEM 

Because of a lack of proficiency in English, many for- 
eign students in the United States have difficulty in making 
a satisfactory adjustment to not only their academic work 
but also to their daily associates. It is hardly an exag- 
geration to say that during the foreign students* sojorn 
in the United States, everything hinges on his ability to 
communicate adequately -- with his teachers, his books, 
his fellow students, and his associates in daily life. 

Most of what he learns must be filtered through a communi- 
cation process, and good communication provides the setting 
in which other problems of adjustment are most easily 
solved, while blocked or distorted communication can give 
rise to a vicious spiral of other personel difficulties. 

Edward Cieslals stated, in his study of Foreign Student 
in American Colleges , that of a dozen qualifications which 
collegiate institutions consider desirable for the admis- 
sion of foreign students, sufficient mastery of English to 
enable them to carry a full program is rated most 
important [Ref. 1] . 

A Department of Defense Study in 1965 determined that 
53 percent of the military students programmed annually for 
technical/professional training in the U.S. service schools 
had English comprehension below the level required for 
successful understanding of course offerings. This did 
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not mean that the students were necessarily given failing 
grades [Ref. 2] . 

The brief discussions which have been presented thus 
far give rise to a number of questions such as. How can one 
measure or determine adequacy of English capability for 
foreign students being educated in the United States?, To 
what degree is a student's success in a given course based 
on his comprehension of English?, Can a student's level of 
English comprehension provide some basis for predicting his 
success in a formal training course?. Are the objectives of 
foreign training being achieved? 

This thesis is concerned with the problem of investi- 
gating and evaluating the relationship of the English com- 
prehension level of foreign students with their academic 
performance at the Naval Postgraduate School. 

If a relationship between English proficiency and achieve- 
ment in courses were determined, it would be possible to pro- 
vide a means for screening candidates for study and predicting 
their academic success at NPS. 

B. ' ASSUMPTIONS AND LIMITATIONS 

Some assumptions and limitations were taken into account 
during inquiring into the problem. These assumptions and 
limitations are as follows: 

1. As most of the curriculla at NPS are 18 months or 
longer in length, foreign students quite possibly increase 
their English proficiency during their time in the United 
States. The degree of improvement may be different depending 
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upon an individual's dilligence, intelligence, personality, 
motivation, etc., and the additional language proficiency 
could affect his academic achievement. It was assumed, 
however, that the original ECL score of a foreign student 
acquired prior to entry into NFS might be a valid predictor 
of his NFS performance. 

2. Many factors which could affect a foreign student's 
academic success at NFS, such as individual background, 
family status, ethnic and national characteristics, economic 
and social status, and motivation, are ignored, and only 
English proficiency as represented by ECL scores was re- 
lated to academic performance in this study. 

3. The prevailing system of assigning alphabetical 
course grades probably does not effectively discriminate 
between students whose performance is not the same. This 

is because identical grades may be assigned to students even 
though their performance covers a relatively wide range. 
Fluctuations of grading policies from one class to another, 
and from one professor to another, also tend to distort the 
validity and reliability of grades. 

4. Because of admission policies, foreign students 
coming to the NFS all had ECL scores of 80 or higher, the 
frequency distribution of the test scores (ECL) was skewed. 
This restriction of range tends to lower the correlations 
between ECL and other variables [Ref. 3] . 

5. Sampling for this research was limited to only the 
recent two years of graduation classes as descriptions for 
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personal data preceding those classes were not available. 
This limited sampling may not be representative o£ the 
whole universe o£ £oreign students able to come to NPS. 

6. It was hypothesized that the relationship o£ English 
comprehension to academic grades would be stronger in the 
curricula having many verval or administrative type courses 
than in the curricula relying very heavily on mathematics 

or engineering type courses. This hypothesis was examined 
by dividing the sample into engineering and non-engineering 
curricula groups. 

7. It was also decided to investigate whether any di£- 
£erences existed between student's military rank. This was 
examined by comparing the statistical results o£ two rank 
groups, one with ranks o£ Lieutenant and below and another 
with ranks o£ Lieutenant Commander and above. 

C. RELATED STUDIES PERTINENT TO THE PROBLEM 

1. The Army Study - A George Washington University 
Study by Dr. George H. Brown (the Brown Study) was concerned 
with £oreign students in U. S. Army schools [Re£. 4]. The 
objectives o£ the research were to assess the academic 
achievement o£ £oreign students in £our selected U.S. Army 
schools and to investigate the relationship between English 
language pro£iciency and academic achievement. The sample 
included 1,159 £oreign students representing 49 countries 
throughout the world. 

This study revealed that 70.2 percent o£ the £oreign 
students were seriously handicapped by English language 
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deficiency, and if standards for United States students had 
been rigidly enforced, 22 percent of all courses completed 
would have been failed: included in the failures would have 

been 51 percent of the students who had poor English pro- 
ficiency and 15 percent of the students who had excellent 
English proficiency as judged by the instructors. 

The report concluded that the actual educational or 
training objective of the foreign student training program 
was not being met, and that English language deficiency was 
a significant factor in the underachievement of foreign 
students . 

The findings and conclusions of the Brown study \\rere 
later strongly supported by a Department of Defense Study 
of English Language Training of Foreign Students [Ref. 5] . 

2. The Air Force Study - Ralph C. Erchinger conducted 
a study on English comprehension level as a predictor of 
achievement for foreign military in training in the Under- 
graduate Pilot Training Course of the United States Air 
Force [Ref. 6] . He investigated the relationship of 
English proficiency, as measured by ECL, with the academic 
and flying training achievements of foreign students. 

The sample included 169 foreign student pilots trained 
in the Undergraduate Pilot Training T-28 Course from 
January 1965 through September 1966. 

The study did not accept the following hypothesis be- 
cause of the low correlation: ECL may be used to predict 

a foreign student's technical flying training or his final 
flying training grade. However, another hypothesis, ECL 
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may be used to predict a foreign student's final academic 
grade, was accepted, based upon a correlation coefficient 
of .7425, which was sufficiently high to provide for mean- 
ingful prediction of academic grades from ECL scores. 

The study concluded that proficiency in the English 
language, as measured by the ECL, may be considered as an 
important contributing factor in the prediction of academic 
success in Undergraduate Pilot Training. 

3. Other Social Studies - Besides the ECL described 
previously, several tests of English ability have been con- 
structed specifically for use with foreign students enter- 
ing university training. Among these tests are the Test 
of English as a Foreign Language (TOEFL) , the University 
of Michigan Tests (the Lado Test) developed at the English 
Language Institute at Michigan, and the Pennsylvania State 
University English Language Proficiency Test. 

These tests, as predictors of success for foreign stu- 
dents, were evaluated by Chase and Stallings in an Indiana 
University Monograph. Their data were from 526 foreign 
students [Ref. 7] . 

They found that the TOEFL and the Pennsylvania State 
English Language Proficiency Test were not significantly 
correlated with grade point average (GPA) . Correlation 
coefficients from .1963 to .2273 were found to be signifi- 
cant in the research, but these correlations were evaluated 
as too low to have practical effectiveness. 

Chase and Stallings concluded that -the tests predicted 
achievement in only a casual way, and they suggested that 
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there is little prospect for relating scores on tests in 
English with a subsequent GPA in college. 

Walter P. Allen investigated the relationship of Lado 
test scores with first semester grades of ninety students 
at the University of Houston [Ref. 7] . This study showed 
product-moment correlations coefficients between .21 and 
.32, and the study indicated that there is a slight cor- 
relation between the English test scores and grades made 
at the University. The correlations were especially marked 
when only English grades were considered. 

In summation, all the above studies showed a positive 
correlation between a foreign student's English facility 
and his academic adjustment, but the results of these studies 
were not conclusive. They showed that the correlation of 
English proficiency to academic achievement of students at 
military schools was stronger than at universities and 
colleges . 
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III. EXPERIMENTAL DESIGN AND PROCEDURES 



A. POPULATION AND SAMPLE 

1 . Population 

The population for this study was considered to be 
of infinite or indeterminate size, including all foreign 
students who had entered the graduate courses of NPS in the 
past, or who would enter the NPS in the future. The size 
of the population largely depends upon the quota available 
at the NPS and the policies of the United States and the 
individual foreign countries. 

2 . Sample 

Table I shows the number of and percent of foreign 
students graduated from NPS during the period of academic 
year (AY) 1973 and 1974, which depicts approximately 10 to 
15 percent of foreign students in each class of the year. 



Table I. 


Number and percent of foreign 


students 


graduated 




during AY-1973 


and AY- 19 74.* 






Graduate 


Total • Number 


U.S. 


Foreign 


Students 


Class 


of Graduates 


Students 


Number 


Percent 


AY-1973 


710 


641 


69 


10.7 


QTR-I 


122 


106 


16 


15.1 


QTR-II 


189 


164 


25 


15.2 


QTR-III 


169 


165 


4 


2.4 


QTR-IV 


230 


206 


24 


11.7 


AY -19 74 


543 


469 


74 


15.8 


QTR-I 


149 


126 


23 


18.3 


QTR-II 


108 


88 


20 


22.7 


QTR-III 


125 


116 


9 


7.8 


QTR-IV 


161 


139 


22 


15.8 


Total 


1,253 


1,110 


143 


12.9 


*Compiled 


from the roster 


filed in the 


Registrar' 


s Office, 



Naval Postgraduate School. 
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The eight consecutive NFS graduate classes o£ 1st 
Quarter, AY-1973 through 4th Quarter, AY-1974, which cor- 
respond to graduation classes of September 1972 through 
July 1974, constituted the sample representing the indeter- 
minate population of all foreign students. 

Purposive sampling was used because transcripts of 
foreign students having ECL scores recorded on them were not 
available for graduate classes preceding September 1972. 

The sample included 110 foreign students represent- 
ing 17 countries throughout the world as shown in Table II. 
This was the maximum size of sample available at the present 
time, as there were some students from countries where the 
ECL test was not required, or whose transcripts were missing. 



Table II. Number and source of foreign students included 
in the sample: by country 



Country 


No. Students 


Country 


No. Students 


Argentina 


1 


Korea 


10 


Brazil 


6 


Peru 


10 


China 


5 


Portugal 


3 


Chile 


4 


Thailand 


15 


Ecuador 


1 


Turkey 


14 


Greece 


8 


Urguay 


3 


Indonesia 


7 


Venezuela 


3 


Iran 


2 


Vietnam 


17 


Japan 


1 


Total 


110 



The English proficiencies of foreign students as 
measured by ECL scores were obtained from records on file 
at the NFS Military Personnel Office, and the final average 
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Table III. Number of foreign students included in the 
sample: by ranks* 



LT 


LCDR 


CDR 


CAPTAIN 


TOTAL 


56 


36 


15 


3 


110 



*Rank shown is the rank held upon reporting to the 
Naval Postgraduate School 

*Ranks of other services than Navy are converted to 
the Navy equivalent 



Table IV shows the constitution of the sample by 
academic year and curriculum. 



Table IV. Constitution of the sample by class and curricula. 



No. Students 
Graduated in 



Department 


Curricula 


AY-1973 


AY-1974 


Total 


Operations 


Operations 


14 


24 


38 


Research § 


Research 


Administra- 

tive 


Management 


9 


5 


14 


Sciences 


Computer 

Science 


2 


1 


3 




Computer 

Systems 


4 


4 


8 




Management 












29 


34 


63 


Engineering 


Electrical 

Engineering 


15 


18 


33 




Mechanical 


12 


9 


14 




Engineering 


L 






27 


20 


47 


Total 




56 


54 


110 
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QPR values were obtained' from the student performance records 
on file at the Registrar's Office, NFS. Both sets of data 
are listed in Appendix C, using a student identification 
code number randomly assigned to each student to preserve 
anonymity. 

B. TEST AND MEASUREMENTS 

To investigate the relationship of English proficiency 
and academic success of foreign students, two variables were 
used in correlational analysis: the set of ECL scores as 
the independent variable, X, and the set of QPR values as 
the dependent variable, Y. These data were tabulated for 
statistical analyses using two packaged sets of computer 
programs: the Statistical Package for the Social Science 

(SPSS) [Ref. 9] and the Job Library for the Statistical Pro- 
gram (STATLIB) [Ref. 10]. 

The nature of these two variables will be discussed 
below. 

1 . ECL Test 

The ECL test is a primary quality control system 
conducted by the Defense Language Institute at the English 
Language School, Lackland Air Force Base, Texas and at many 
overseas Military Assistance Advisory Group and Missions. 

The ECL is used to screen applicants from non- 
English speaking countries for military training in the 
United States . 

In light of the importance of English as a primary 
communication media for training in the United States, the 
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U.S. Department o£ Defense specifies that a knowledge of 
English language is a prerequisite for foreign students to 
attend schools in the United States and requires that all 
students must obtain passing scores on the English Compre- 
hension Level Test. 

An ECL test takes about 60 minutes to administer, 
and tests reading and listening comprehension. The results 
of the test range from zero to one hundred points which are 
translated into qualification levels as established in a 
numerical system by DLI. For instance, an individual with 
a 70 ECL knows much more than twice the amount of English 
compared to one with a 35 ECL, and the progressive propor- 
tions between a 45 and 90 ECL are not the same as those 
between 35 and 70 ECL. 

The minimum ECL requirements for entry into various 
schools in the United States are determined by the schools 
of each different service and confirmed by the DLI Head- 
quarters. Various levels of English comprehension are de- 
fined as follows [Ref. 11] : 



0 - 39 ECL 
40 - 59 ECL 
60 - 69 ECL 

70 - 79 ECL 



beginning language students 

elementary- intermediate level 

qualification level for low-level 
or apprentice specialized training 

qualification level for beginner 
courses where the language limita- 
tion is critical 



80 ECL and above*: qualification level for professional 

career and advanced courses 

*ECL required for entry into NFS belongs to this 
category 
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The psychometric characteristics of the ECL tests 
are shown in Appendix A. 

2. Academic Success at NFS 



A student's academic performance 


at the NPS is 


evaluated on the basis 


of a quality point 


number assigned 


letter grade achieved 


in a course as follows [Ref. 12] : 


Performance 


Grade 


Point Value 


Excellent 


A 


4.0 




A- 


3.7 




B+ 


3.3 




B 


3.0 




B- 


2.7 




C + 


2.3 




C 


2.0 




C- 


1.7 




D+ 


1.3 




D 


1.0 


Failing 


X 


0.0 



IVhen the quarter hours value of a course is multi- 
plied by the quality point number of the student's grade, 
a quality point value for the student's work in that course 
is obtained. 

The sum of the quality points for all courses di- 
vided by the sum of the quarter hour value of all courses 
gives a weighted numerical evaluation of the student's per- 
formance termed the Quality Point Rating (QPR) . 

A student achieving a QPR of 3.0 has maintained a 
"B" average in all courses undertaken. 

Academic success at NPS is defined as the successful 
completion of all courses of a curriculum, supported by a 
final minimum QPR of at least 3.0, and an approved thesis, 
if required. 
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C. ANALYTICAL PROCEDURES 

1 . Correlation Analysis 

To investigate the relationship between the two 
variables, ECL and QPR, regression and correlation analyses 
was conducted. A standard statistical technique for evalu- 
ating the effectiveness of test scores for predicting achieve 
ment is the correlational validity coefficient. 

Correlation coefficients provide indices of the ex- 
tent to which excellence on one measure is associated with 
excellence on another measure, or, in other words the de- 
gree to which variables or measures vary together. Mathe- 
matically, the value of the coefficient may range from a 
perfect positive correlation (+1.0), through no relationship 
(0.0), to a perfect negative correlation (-1.0). 

In general, when correlating variables, the coeffi- 
cients indicate roughly the following strengths of relation- 
ships between the variables [Ref. 12] 



Less than .20 

.20 - .40 

.40 - .70 

.70 - .90 
.90 - 1.00 



Slight, almost negligible relation- 
ship 

Low correlation, definite but slight 

relationship 

Moderate correlation, substantial 

relationship 

High correlation, marked relationship 

Very high correlation, very depend- 
able relationship. 



However the interpretation of the value of a correla- 
tion coefficient is purely relative to the circumstances 
under which it \^as obtained and should be interpreted in the 
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light of those circumstances. Many who have employed tests 
for vocational guidance or vocational selection have fol- 
lowed a tradition that the minimum validity coefficient for 
a test to be practically useful is about .45 [Ref. 13]. 

Recent experiences have shown that this standard 
can be too rigid. According to McKenna, it is noted that 
most validity coefficients developed by personnel research 
range from .25 to .50. Yet these validities can have prac- 
tical value, and it is unusual to have a validity coeffi- 
cient of more than .50 [Ref. 14]. 

The correlation coefficients for this study "were 
calculated by SPSS using the Pearson product-moment method. 
Statistical results are shown in the tables and figures of 
Chapter IV. 

2 . Regression Analysis 

IVhile the correlation coefficient is a useful index 
for evaluating the correlation between variables, this in- 
formation cannot give a clear picture of the predicted aca- 
demic achievement of individual foreign students who scored 
at various levels on the ECL test. 

However, through the use of regression analysis, 
such predictions can be accomplished by establishing a line 
of regression for ECL test scores versus academic grades. 
This linear function can be expressed as an albegraic equa- 
tion Y = a + bX, which provides the estimate of an unknown 
variable Y when the value of another variable X is known 
[Ref. 15]. 
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For each case of analysis, a scatter diagram was 
obtained with the dependent variable plotted on the ordinate 
and the independent variable along the abscissa to check 
for any curvilinear relationships, but none were found. 

The linear line of least squares was computed and plotted 
from the formula Y = a + bX by substituting coefficients "a" 
and "b”. 

For the purpose of summarizing and displaying graph- 
ically the probability or likelihood of individual student's 
academic success, an individual expectancy chart was drawn 
from the scatter diagram. An expectancy chart is a graphic 
display of presenting the probability of attaining success 
associated with a given score range [Ref. 16] . 
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IV. ANALYSIS OF THE RESULTS 



A. ANALYSIS OF DATA 

1 . Relation of ECL to QPR 

Table V contains a statistical summary of ECL and 
QPR. Considering the total sample, the validity coefficient 
of correlation was found to be .2897, which is significant 
at the .001 level. 

To check for the reliability of the statistical 
results, these results were cross validated by dividing the 
sample into 56 students of the AY-1973 class as a develop- 
mental group and 54 students of the AY-1974 class as a 
validation group. The AY-1973 class showed a positive cor- 
relation of .3100, which is slightly higher than the coef- 
ficient .2626 for AY-1974. 

Table V. Statistical summary of ECL and QPR. 





AY-1973 


AY-1974 


Total 


Number of sample 


56 


54 


110 


ECL Mean 


86.96 


86.33 


86.65 


S.D. 


5.93 


4.94 


5.45 


Range 


80-100 


80-100 


80-100 


QPR Mean 


3.17 


3.13 


3.15 


S.D. 


0.40 


0.39 


0.39 


Range 


2.25-3.97 


2.25-3.97 


2.25-3.97 


Correlation Coefficient 


0.31 


0.26 


0.29 


Standard Error of Estimate 


0.38 


0.38 


0.38 


Confidence Band: Upper 


3.55 


3. 51 


3.53 


Lower 


2.79 


2.75 


2.77 


Regression Intercept "a" 


1. 37 


1.32 


1.34 


Regression slope "b" 


0.02 


0.02 


0.02 


Figure 1 shov>:s the 


estimated 


regression 


line and 



associated confidence band. 
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REGRESSION LINES 




^Regression coefficients a and b were significant 
at < .001. 



Figure 1. Estimated line of regression based upon 110 
foreign students. 
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The regression line was determined by substituting 
computed values of regression coefficients a = 1.3373 and 
b = .0209 in the general equation Y = a + bX, and computed 
values of Y were determined and plotted as a solid line in 
Figure 1. The confidence band in the Figure 1 was estab- 
lished as plus and minus one standard deviation (standard 
error of estimate) and is represented by the dotted lines. 

To show an individual's chances of academic success 
at NFS, the relevant data were tabulated and presented 
graphically as an expectancy chart in Figure 2, and as a 
tabulation in Table VI. 



EXPECTANCY CHART 



ECL SCORE 


CHANCES IN 100 OF BEING SUCCESS 


% 


95 - 100 




100 




90 - 94 




58.8 


S^\\\\\\\^ 




85 - 89 




73.0 


^\\\\\\\\^ 


80 - 84 




65.2 


1 1 f 



25 



50 



75 



100 



Figure 2. Expectancy chart for individual prediction of 

academic success based on 110 foreign students. 
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Table VI. Tabulation of the scatter diagram of ECL and QPR based on N = 110. 



if) 

in 


o 


to 


o 


o 


o 


o 


o 


o 


O 


m <D 


• 


• 


• 


• 


• 


• 


• 


• 


• 


o o 


1 o 


to 


o 


O 1 


1 o 


o 


o 


o 


o 


o 


vD 


to 


o 


o 


o 


o 


o 


o 




;3 

CO 






T — 1 


T — 1 


T — 1 


T — 1 


T — 1 


rH 





O rH 

•HOCs3<^OOOOOOOK) 

• Ctf to 

O Ph 



m in 

O if) 
<D 

u 

• u 

o :3 

^ CO 



o to 



CNJ to CNJ o to 



to 

!>. 



(0 

<D 

CJ o 

w o 

CO 



if) 

if) 

m <D 
o o 
u 

CO 



O 1-H 
•H 
• 03 
O Ph 
12; 



03 



rH 


CV3 


to 




LO 


VO 




CO 


Oi 


o 


+-> 


Oi 


Oi 


Oi 


CJ) 


cn 


cn 


O) 


Oi 


CTi 


o 


o 




















rH 


H 


O 


rH 


o 




VO 




o 


o 


CNI 


LO 




LO 






VO 


CO 


LO 


o 


o 


Oi 




VO 


VO 


LO 




VO 




CO 


VO 




VO 


CO 


VO 




to 


rsj 


T — 1 


to 


rH 


Csl 


to 


to 


T — 1 


T — 1 



m (A) 
o C/l 

<D 

U to ^ o CSI 

• U rH 

o :3 

12; CO 



VO to csi csi 



if) 

<D 

^ U 
a o 
w o 

CO 



o 

oo 



CVJ 

CO 



tO’^Lnvoi>>coaiO 

cocococococococn 



31 



w 



I 

I '4 




2 . Comparison of Curricula Subgroups 

A comparison o£ the statistical results of the two 
major subgroups combining similar families of curricula was 
made in an attempt to see if there were any significant dif- 
ferences between engineering and non-engineering curricula. 
Table VII is a tabular summary of the relationship o£ ECL 
test scores and QPR values of the two different groups. 



Table VII. Comparisons of statistical results of curricula 
subgroups . 



Subgroups by Curricula 



OR/AS Dept.* Eng. Dept.** 



Sample numbers 63 

ECL(X); Mean 87.2380 

S.D. 5.2570 

QPR(Y) ; Mean 3.0967 

S.D. 0.3880 

Correlation Coefficient 0.3289 

Standard Error of Estimate 0.3694 



47 

85.8723 

5.6554 

3.2174 

0.3932 

0.2978 

0.3795 



* OR/AS means Operations Research/ Administrative Sciences 
**Eng. means Engineering 



The mean ECL scores of students in OR/AS Department 
is 87.2380, which is slightly higher than 85.8723 of engineer- 
ing curricula, while the final average QPR obtained by the 
students in OR/AS Department, 3.0967, is lower than the 
3.2174 obtained in engineering curricula. 

The correlation of ECL test scores with QPR values 
in the OR/SA Department was .3289. This is slightly higher 
than the .2978 in engineering curricula. 

The disparities between the results for these two 
groups are also depicted in Figure 3,- with the estimated 
regression line of ECL and QPR for the OR/AS Department as 
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REGRESSION LINES 




Note: Regression coefficients a and b were significant 

at < .02. 



Figure 3. Estimated regression lines of two curricula sub- 
groups (OR/AS Dept. N = 63, ENG Dept. N = 47). 
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a solid line and that o£ the Engineering Department as a 
dotted line. 

Individual expectancy of academic success, as shown 
in Figure 4, is higher in the engineering curricula for 
student's whose ECL scores were under 90. 



EXPECTANCY CHART 



ECL SCORE 


CHANCES IN 100 OF BEING SUCCESS* 


% 








95 - 100 




100 






100 






X u u 


90 - 94 




61.5 
SO . 0 


85 - 89 




69.6 






78.6 


80 - 84 




54.6 






75.0 





0 25 50 

OR/AS Department Curricula 



75 



100 



ENG. Department Curricula 

* QPR > 3.00 



Figure 4. Expectancy chart for individual prediction of 
academic success in two curricula subgroups 
(OR/ AS Department N = 63, ENG. Department N = 47) 
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3 . Comparison of Rank Subgroups 

In an attempt to see if there were any significant 
differences in the ECL-QPR relationship between foreign 
student’s military ranks, a comparison of the statistical 
results of the two groups of rank, Lieutenant (LT) and be- 
low versus Lieutenant Commander (LCDR) and above, was con- 
ducted. Table VIII shows the results for the two groups. 



Table VIII. Comparisons of statistical results of rank sub- 
groups . 



Subgroup by Rank LT and Below LCDR and Above 



Sample Numbers 56 

ECL(X) ; Mean 86.0357 

S.D. 5.4137 

QPR(Y) ; Mean 3.1148 

S.D. .4140 

Correlation Coefficient .2336 

Standard Error of Estimate .4063 



54 

87.2963 
5.4587 
3.1830 
. 3707 
. 3398 
. 3520 



Those students with higher rank, LCDR and above, had 
a slightly higher mean ECL and mean QPR than those of lower 
rank, LT and below. 

The correlation coefficient of ECL test scores with 
QPR values of the higher rank students is .3398, which is 
higher than the .2336 for the group of lower ranking students. 

Regression lines of ECL and QPR for these two groups 
by rank are shown in Figure 5, with a solid line for LT and 
below, and a dotted line for the LCDR and above group. 

Individual expectancy of academic success in the 
tv^ro groups by rank is graphically presented in Figure 6. 

In addition, the probability of academic success in accordance 
with ECL cut-off scores established for selection purposes 

is shown in Figure 7. 
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REGRESSION LINES 




Note: Regression coefficients a and b were significant at 

< . 04 . 



Figure 5. Estimated Regression Lines of Two Rank Subgroups 
(Based upon LT and Below N = 56 , LCDR and above, 
N = 54.) 
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EXPECTANCY CHART 



ECL SCORE 


CHANCES IN 100 OF BEING SUCCESS* 


1 


95 - 100 




100 

100 


iXI\ \ \ \ \ \'\ \A \ \ \ \ WWXWV 






90 - 94 


vYWWWWW' 


50.0 

63.3 






85 - 89 




57.1 

81.8 


v\\\\\\\\\\\\\ 








80 - 84 




70.0 

52.9 


VWWWWWWWW^^ 











0 25 50 75 100 



LT and below 
LCDR and above 



* QPR > 3.00 






Figure 6. Expectancy Chart for indivisual prediction of 
academic success in two rank subgroups. 
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Probability of Success 




O 

o 



CO 

cn 



o 



cn 



CV3 

cn 



o 

cn 



CO 

CO 



VO 

CO 



CO 



CNI 

CO 



o 

CO 



o 

o 



o o 

cn CO 



o o o 

VO 



CO 

<D 

O 

o 

CO 

t+H 

m 

o 

t 

+-> 

13 

U 

u 

w 



38 



Figure 7. Probability of academic success Based upon ECL cut-off scores as established 
between 80 ECL and 100 ECL. 

(OR/ AS Dept. N = 63, ENG Dept. N = 47, Total N = 110.) 



B. EVALUATION OF STATISTICAL RESULTS 



As has been stated previously, the statistical data for 
the present analysis was generated from the sample of 110 
foreign students. The frequency distribution of the academic 
grades (QPRs) was very close to a normal distribution, with 
a range of socres from 2.25 to 3.97, while that of the test 
scores (ECL) were skewed to the left toward 80, because the 
score of 80 was originally used as cut-off for minimum 
English language proficiency. 

As shown in Table V, the correlation of ECL with QPR 
for the total sample was only .29, which is a definite but 
small relationship. 

The standard error of the estimate was computed to be 
.3779, meaning that an estimated academic grade plus and 
minus .3779 would contain 68.27 percent of the true values. 
Using this confidence level, as shown in Figure 1, we can 
say we are 68 percent confident an individual's QPR will 
fall within the indicated band based upon his ECL test 
score, or, in other words, the individual has an 84 percent 
probability of achieving a QPR equal to or greater than 
that QPR indicated by the intersection of the lower confidence 
limit (minus one standard error) , and his achieved ECL test 
score. However, based upon the total sample, the correla- 
tion of ECL test scores with foreign student's QPRs is not 
particularly high. This correlation was probably lowered 
by a restriction of range on the ECL scores. As shown in 
Figure 2, if an English proficiency of a foreign student 
were 95 ECL and above, he would have a perfect chance of 
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succeeding, or, otherwise, his chances of success would be 
around two out of three. 

Analyses were also made by dividing the total sample 
into two different curricula subgroups: OR/AS and engineer- 

ing. As depicted in Table VII and Figure 3, the comparison 
of the statistical results for these two subgroups reveals 
that English language proficiency is more predictive of 
performance in non-engineering curricula than in an engineer- 
ing curricula. We may, therefore, expect that by using data 
generated for the two different subgroups, English proficiency 
for the student in administrative science type curricula 
should be higher than those in the engineering type curricula 
to achieve the same academic grades. Figure 3 bears out this 
belief. 

Analyses were also conducted to compare the statistical 
results of subgroups by dividing the student's military ranks 
into two groups, LT and below and LCDR and above. The re- 
sults of the analysis given in Table VIII shows that those 
foreign students with higher rank had a stronger correlation 
of ECL to QPR than did the lower rank group. This is evalua- 
ted as meaning that if present student assignment procedures 
were continued, students with ranks of LCDR or above would 
be expected to be more successful at NFS than officers of 
lower ranks . 

In addition, further analysis were intended to check for 
effectiveness of different ECL cut-off scores for entry 
into NFS. Visualizing Figure 7, if the cut-off score for 
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entry into NPS, for instance, were set at 80 ECL and above, 
as it is at the present time, the probability of academic 
success would be 65.1 percent in OR/AS curricula, 76.6 
percent in engineering curricula, and 70.0 percent as a 
whole . 

The chart shows that these probabilities of academic 
success do not change immensely until the ECL cut-off score 
reaches 93 ECL and above, but at the cut-off score of 94 
ECL and above the probability of academic success goes up 
to one hundred percent in both groups. In other words, 
the minimum ECL which would have one hundred percent chance 
of success was found to be 94, and no significant differences 
in success rates were found between scores of 80 and 93. 

This means that if the ECL cut-off score for selection of 
NPS candidates was set at 94 ECL and above, there would be 
no academic elimination at all. 

Results of this study give some support to the effective- 
ness of the cut-off score of English comprehension level as 
measured by ECL and confirms that those who acquire ECL of 
80 and above more often than not have the proficiency in 
English needed for success at NPS. 

C. RELATED STUDIES COMPARED 

As h.as been discussed in Chapter II. C, most studies of 
English proficiency have shown a relatively low correlation 
with the academic performance of foreign students in the 
United States. 

As seen in Table IX, the probability of academic success 
of foreign students at NPS is 70 percent, which is comparable 
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to the 78 percent of the Brown study [Ref. 18] conducted 
at Army schools. The correlation coefficients of the pre- 
sent study of around .30 is comparable to, or slightly 
higher, than most of the previous studies except the un- 
usually high correlation of .7425 found in the Erchinger 
study [Ref. 19] . At this point, the reader should recall 
that the correlation between ECL and QPR suffered in this 
study probably because of the restriction of range on ECL. 



Study By 
Chase § 



Table IX. Summary of Analogous Study Results. 

Test Achievement Correlation 



Subj ects 

Indiana University 



Stallings 


N = 526 


Lado 


GPA 


. 1963- 










.2273 


Allen 


University of 




GPA 


.2100- 




Houston N = 90 


Lado 




. 3200 


Brown 


Army Schools 


Judge 


Pass/ 






N = 1159 


by In 


Fail 


OO 






s true 


- 








tors 






Erchinger 


Air Force Under- 


ECL 


Flying Train- 






graduate Pilot 




ing Academic 


. 1970 




Training N = 169 




Grade 


. 7425 








Final Grade 


. 1729 


Present 


NPS 


ECL 


QPR 




Investigator 


Total N = 110 




QPR 


.2897 




0R8AS N = 63 




QPR 


. 3289 




ENG. N = 47 






. 2978 


The disparity in the predictive 


power 


of an English 




language test 


between verbal and written ^ 


course work and 




technical instruction found in the 


Erchinger study, is mere 


or less in agreement with the findings of 


the present study 



revealed in Table IX. 
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V. CONCLUSIONS AND RECOMMENDATIONS 



A. CONCLUSIONS 

This study was undertaken to investigate and evaluate 
the relationship of English comprehension levels of foreign 
students with their graduate academic performance at NFS. 

To this end, ECL test scores of the foreign students 
obtained prior to the entry into NFS and their final QFR 
scores were used as variables for regression and correla- 
tion analysis. 

The results obtained from the present study revealed a 
positive correlation of .2897 which is a statistically 
significant but marginally useful relationship between the 
ECLs and QFRs . The relationship was found to be slightly 
different for engineering, as compared to non-engineering 
curricula. It was stronger for students in the OR/AS cur- 
ricula than for those in the engineering curricula, which 
means that some variables other than English proficiency, 
such as pre-knowledge in this field, strongly influence the 
academic performance of students in engineering curricula. 

In addition, the comparison of the QFR-ECL relationship 
within different rank groups showed that foreign students 
with military ranks of LCDR and above obtained higher ECL 
test scores and higher final QFRs than those with ranks of 
LT and below. The relationship of ECL to QFR was also found 
to be higher for foreign students with higher ranks than for 
those with, lower ranks. 
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Because the ECL scores were originally cut off at 80 or 
above, the predictive power of ECL as assessed by means of 
the ECL-QPR correlation was constrained by restriction of 
range . 

However, the results of this study showed a 1.0 prob- 
ability of academic success for those students who had re- 
ceived a 94 ECL or above prior to entry into NPS. 

In general, the correlation coefficients which were 
found in the present study were not particularly high, but 
the results have to be judged on the basis of values ordi- 
narily obtained in correlational analysis of this kind 
where restriction of range has operated. 

B. RECOMMExNDATIONS FOR FUTURE STUDY 

1. It is recommended that future studies investigating 
the relationship of English language problems and the aca- 
demic performance of foreign students in the Naval Post- 
graduate School, group the subjects by curriculum or group 
courses of study according to the characteristics of their 
contents, verbal and quantitative. 

For those who may be interested in finding other pre- 
dictors for the academic success of foreign students such 
as "age,” "number of years after baccareaurate study," 
number of years in military service," "grade record and 
source of baccareaurate study," "military academy or civilian 
college," and "preschool experience of graduate level" may 
be useful predictors. 

It is also recommended that the Naval Postgraduate School 
or interested students consider administering the Graduate 
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Record Examination (GRE) test to foreign students before or 
around the time they begin to study at the NFS and examine 
the GRE score as a possible predictor for academic perform- 
ance by the follow-up method. The GRE test scores has been 
verified as an effective predictor for academic success of 
graduate level study by many earlier studies [Ref. 20] . 

If the GRE test scores were a good predictor, it might 
be suggested that the ECL test be replaced by the GRE test 
for the selection of foreign students for graduate study at 
the Naval Postgraduate School. 
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APPENDIX A: PSYCHOMETRIC CHARACTERISTICS OF THE ECL TESTS* 



The statistical data of the series 6500 test forms A-F 
are as follows: 





A 


B 


C 


D 


E 


F 


Maximum Possible Score 


100 


100 


100 


100 


100 


100 


Mean Score 


60.3 


59.5 


62.0 


60.0 


62.3 


62.6 


Standard Deviation 


20.9 


20.6 


20.3 


20.7 


20.1 


23.5 


Mean Difficulty Index 


.62 


.62 


. 63 


.62 


.64 


.64 


Mean Validity Index 


.53 


.51 


.51 


.52 


.48 


.57 


Reliability Index for 














Internal Consistency 


.95 


.95 


.95 


.96 


.94 


.97 


(Kuder- Richards on 














formula #21) 














Alternate-form Reli- 
ability Coefficient 


AxB 


BxC 


BxD 


DxE 


ExF 




(Pearson Produce - 
Moment Correlation 


.94 


.95 


.96 


.95 


.96 





formula) 

*1. Source: Information on ECL Tests, DLIEL-D, Defense 

Language Institute, Department of the Army, 

26 August 1974 

*2. Validation Procedure 

Six preliminary forms of the ECL tests were constructed 
with 150 questions in each form. After presenting them to 
1,200 Army, Navy and Air Force students representing 35 dif- 
ferent countries, each form was revised and shortened to a 
100-item test. Each item was checked for content validity, 
difficulty level and discriminating power. The items represent 
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a sample of material found in the American Language Course. 
The 6500 ECL series were correlated with the 6200 ECL series, 
and the correlation coefficients ranged from .93 to .95, in- 
dicating a high correlation between the two series. 
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APPENDIX B: LIST OF SAMPLE ECL AND QPR SCORES 



dent 

ntification 

her 


Graduate 

Academic 

Year 


ECL 

Test 

Curriculum Score 


Total 

Average 

QPR 


101 


72-73 


O.R. 93 


2.60 


102 


t1 


" 88 


3.03 


103 


rr 


" 95 


3. 35 


104 




" 84 


3.23 


105 


M 


" 92 


3.65 


106 


11 


” 86 


2.71 


107 


11 


" 92 


3.15 


108 


11 


" 93 


2.91 


109 


11 


" 80 


3.34 


110 


11 


" 85 


2.48 


111 


11 


" 82 


2.42 


112 


11 


” 81 


2.43 


113 


11 


" 84 


2.50 


114 


11 


” 84 


2.32 


115 


11 


General 85 


3.60 






Management 




116 


11 


" 100 


3.65 


117 


11 


” 94 


3.57 


118 




" 89 


3. 37 


120 


M 


” 80 


3.40 


121 


11 


" 85 


3.18 


122 


11 


" 80 


3. 27 


123' 


11 


" 90 


3.33 


124 


11 


Computer 89 


3. 17 






Management 




125 


11 


” 84 


3.10 


126 


11 


" 81 


3. 21 


127 


11 


" 86 


3.24 


128 


11 


M 94 


3.50 


129 


11 


” 87 


3.12 


201 


11 


Electrical 88 


2.25 






Engineering 




202 


11 


" 98 


3.54 


203 


M 


” 84 


3. 87 


204 


11 


” 93 


3.47 


205 


11 


” 86 


3.09 


206 


11 


” 80 


3.11 


207 


11 


" 92 


2.90 


208 


11 


" 80 


3.09 


209 


11 


’’ 97 


3.71 


210 


11 


” 97 


3.93 


211 


11 


” 88 


3.11 


212 


11 


" 81 


3. 15 
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Student Graduate 
Identification Academic 
Number Year 



213 72-73 

214 

215 " 

216 

217 

218 

219 . ” 

220 " 

221 " 

222 

223 

224 

225 

226 

227 •' 

301 73-74 

302 '• 

303 

304 

305 

306 

307 •' 

308 

309 

310 

311 

312 

313 

314 

315 

316 '• 

317 

318 •• 

319 

320 

321 

322 

323 

324 " 

325 •• 

326 '• 

32 7 ” 

328 ” 

xoq «« 

330 •• 



Curriculum 


ECL 

Test 

Score 


Total 

Average 

QPR 


Electrical 


80 


2.84 


Engineering 




81 


2.91 


n 


84 


3. 26 


Mechanical 


84 


3.49 


Engineering 


n 


95 ■ 


3.14 


M 


80 


2.86 


tt 


80 


3.33 


!! 


86 


3.34 


tt 


80 


3. 16 


tt 


88 


3.24 


tt 


80 


3.02 


tt 


93 


2.49 


tt 


89 


3.49 


tt 


83 


3. 71 


tt 


80 


3.34 


O.R. 


80 


2.88 


tt 


84 


3. 33 


tt 


93 


2.90 


tt 


89 


3.97 


tt 


83 


3. 16 


tt 


83 


2.80 


tt 


97 


3.43 


tt 


87 


2.92 


tt 


99 


3.07 


tt 


85 


3.52 


tt 


90 


2.92 


tt 


81 


3.01 


tt 


86 


2.90 


tt 


92 


2.68 


tt 


80 


2.90 


tt 


92 


3.12 


tt 


94 


3.48 


tt 


89 


2.92 


tt 


80 


3.09 


tt 


89 


3. 10 


tt 


84 


2.34 


tt 


82 


2.45 


tt 


88 


2.57 


tt 


88 


2.25 


General 


85 


3.21 


Management 


It 


88 


3.54 


If 


88 


3.47 


tt 


89 


3.41 


tt 


90 


3.35 


Computer 


80 


2.76 


Management 
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Student 

Identification 

Number 


Graduate 

Academic 

Year 


Curriculum 


ECL 

Test 

Score 


Total 

Average 

QPR 


331 


73-74 


Computer 

Management 


84 


3.58 


332 


t? 


If 


85 


3.28 


333 


M 


If 


88 


3.28 


334 


tl 


If 


81 


3.07 


401 


!? 


Electrical 

Engineering 


88 


3.61 


402 


ft 


If 


80 


2.40 


403 


f f 


fl 


89 


3.21 


404 


f! 


If 


84 


3.29 


405 


M 


If 


93 


3.29 


406 


If 


If 


87 


2.94 


407 


If 


If 


88 


3.57 


408 


If 


If 


84 


3.23 


409 


If 


If 


80 


3.38 


410 


If 


If 


80 


3.49 


411 


If 


If 


84 


3.07 


412 


If 


If 


87 


3.20 


413 


If 


If 


88 


3.89 


414 


If 


If 


81 


2.83 


415 


If 


If 


85 


3.49 


416 


If 


If 


80 


3.01 


417 


If 


If 


87 


2.81 


418 


If 


If 


100 


3.51 


419 


If 


Mechanical 

Engineering 


80 


2.33 


420 


If 


If 


84 


3.83 
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